The present invention relates to augmented reality of displayed video streams and more specifically, to synchronizing an augmented reality video stream with a displayed video stream.
Augmented reality (AR) techniques are becoming increasingly popular particularly using handheld devices such as mobile phones and tablets as well as AR glasses.
AR techniques are used to overlay AR video streams including images and information onto displayed video streams. The displayed video stream may be displayed by a separate device, for example in a cinema. Therefore, the AR video stream and the displayed video stream are not linked making it difficult to provide synchronization between the AR video stream and the displayed video stream.
Current AR techniques do not handle video streams very well. The way AR systems work is that they either take an anchor point in a scene by using object recognition to match an object that can be seen to a dictionary of markers, or detect a marker in a scene and determine from the content of the marker the correct media element to display on top of that marker. When using an AR device to look at a video feed that includes changes of scene, the markers will be lost and the device will need to calculate new anchor points at each scene. This does not allow full scale AR overlays onto a video stream, as it is too hard with current technology to use a video feed as a marker.
In addition, a user may be at an angle to the displayed video which may distort the images in the displayed video which provides further challenges when overlaying an AR video stream.
The displayed video stream and an AR video stream may be played with the same speed with the aim of automatically keeping in sync. However, they may get out of sync for two reasons.
Firstly, different devices may have different hardware and software drivers, and this will inevitably lead to some difference over time.
Secondly, when there are separate controls for the displayed video stream and the AR overlay video stream, the AR user might pause/resume etc. and hence the need for further synchronization mechanisms.
Therefore, there is a need in the art to address the aforementioned problems.